Apache Spark 2.x Cookbook by Rishi Yadav

Author:Rishi Yadav [Yadav, Rishi]
Language: eng
Format: azw3
Tags: COM062000 - COMPUTERS / Data Modeling & Design, COM021040 - COMPUTERS / Databases / Data Warehousing, COM018000 - COMPUTERS / Data Processing
Publisher: Packt Publishing
Published: 2017-05-31T04:00:00+00:00

Taking a closer look at Structured Streaming

Structured Streaming has been introduced in various places in this chapter, but let's use this recipe to discuss some more details. Structured Streaming is essentially a stream-processing engine built on top of the Spark SQL engine.

An alternative way to look at streaming data is to think of it as an infinite/unbounded table that gets continuously appended as new data arrives.

The four fundamental concepts in Structured Streaming are:

Input table: To input the table

Trigger: How often the table gets updated

Result table: The final table after every trigger update

Output table: What part of the result to write to storage after every trigger

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.